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DYNAMIC ANOMALOUS PIXEL DETECTION AND CORRECTION 

BACKGROUND OF THE INVENTION 
The present invention relates to digital still or video cameras using a solid 
5 state image pickup device and more particularly to methods and systems used to detect 
and correct defective pixels obtained from a solid state image pickup device. 

The foundation of digital imaging is the image sensor. Just as in a 
traditional camera, light enters a digital still or video camera through a lens controlled by 
a shutter. When the shutter opens, rather than exposing film, the digital camera collects 
10 light on an image sensor - a solid state electronic device. Image sensors contain a grid of 
microscopic photosites or pixels. In a camera, each photosite on the image sensor may 
represents one pixel of a digitized image. Two types of image sensors that are quite 
prevalent in the digital imaging field are the charge-coupled device (CCD) and the 
complimentary metal oxide semiconductor (CMOS) image sensor arrays. These image 
15 sensors may contain anywhere from a few hundred to millions of photosites to convert 
light that is shining on them into small electric currents which can be electronically 
processed and interpreted and eventually used to recreate and display a target image on a 
display device. 

The processes used to produce these image sensors though highly 
20 advanced are not perfect. Photosites in an image sensor array can vary in their sensitivity 
or ability to convert light into electric currents. Some variation is expected and tolerable, 
but on occasion, individual photosites in an array can be dead, inactive or over active. 
This can result in a displayed image having a dark or black dot, bright white dot or 
incorrect color dot, all of which are unwanted and are undesirable to the end user. Those 
25 in the digital imaging field have also recognized the problems caused by improper 

digitization of target images and have proposed various solutions to detect and correct 
defective pixels. These prior art methods and systems can be broadly classified into two 
groups. 

On one hand, there are various hardware-based methods for detecting and 
30 correcting defective pixels taken from an image sensor array. These hardware-based 
methods and systems are quite common to video camera manufacturers. Most 
conventional video cameras using a solid state image pickup device, incorporate a 
defective pixel detection and correction circuit for correcting defective pixels taken from 



an image sensor array. The defective pixels are produced when or after the image sensor 
array is manufactured. In such a camera, a defective pixel correction circuit detects a 
defective pixel and stores the position data and various data related to the defective pixel 
in a read only memory (ROM) or the like. Then, when the video camera is in use, pixel 
5 data from the defective pixel is replaced by data from a pixel near the defective pixel. 
One such method and system is disclosed in U.S. Pat. No. 5,796,430. The disadvantage 
of such approaches is the need to incorporate and program memory devices during 
product assembly and testing which add costs and delays. Additionally, the hardware 
device costs will also increase due to the need to add the correcting circuitry or logic to 
10 the video processing application specific integrated circuit (ASIC). 

On the other hand, there are various software-based methods for detecting 
j 3 and correcting defective pixels in digital images. One such method and system is 

: p disclosed in U.S. Pat. No. 5,982,946. Such software-based methods are generally aimed 

: %] at correcting bad image pixels in an already digitized and fully processed image. These 

W 15 already digitized images are in their final displayable and stored forms which have 
g already been through color processing, compression and various other processing that are 

:\ performed on the raw data which is read off an image sensor array. Therefore, such 

; y software-based methods which treat completely processed digital images, may correct 

l n anomalous pixels that are caused by any number of artifacts such as dust or dirt on the 

5j 20 original scene, dust or dirt on the lens which was used to capture the scene as well as 
anomalous pixels in the digitized image which were caused by defective photosites. 
These methods typically rely on rather sophisticated and expensive systems and computer 
software to detect and correct bad pixels. These methods generally require user input to 
detect the location of the potentially bad image pixels. Once the user has visually 
25 scanned an entire image and flagged potentially bad image pixels, the system 

incorporating the software program takes over to correct the flagged defective pixels. 
Besides requiring user input, such approaches are also expensive, tedious and very time 
consuming. 

The proliferation of inexpensive PC-interfacing digital still and video 
30 camera devices requires rapid, dynamic, inexpensive and intelligent defective pixel 
detection and correction solutions. 
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SUMMARY OF THE INVENTION 
The present invention provides a method and a system for dynamically 
detecting and correcting anomalous pixels in the raw data taken from an image sensor 
array such as a CCD or a CMOS sensor array, thus allowing the use of dumb cameras to 
5 capture digital images for subsequent use by an intelligent host - such as being displayed 
on a computer monitor. This invention uses software algorithms running on an intelligent 
host processor to dynamically correct the anomalous pixels in the raw data taken from an 
image sensor array typical of those in a digital still or a video camera. The raw image 
sensor data is the unprocessed brightness data output obtained from the image sensor and 

10 which has not gone through any lossy compression or color processing. The image 

sensor reads analog voltage or current, converts to digital and sends the signal to a host 
with no further processing or compression. The photo sites on the image sensor are used 
to capture either color or monochrome digital still or video images. The raw image data 
is sent to the intelligent host over a bus with a data transfer rate which is determined by 

1 5 the bus protocol of the particular bus such as a universal serial bus (USB) or a parallel 
port. All image processing is moved from the camera to the intelligent host. Using the 
combination of a dumb camera which provides raw data to an intelligent host, which does 
all the subsequent image processing, the system works by scanning an image frame for 
pixels that vary more than a specified amount in their brightness value from their 

20 neighboring pixels and designating those as defective pixels. The location and frequency 
of the photosites sending the defective pixels are stored in a statistical database in the 
computer's memory. The brightness value of a defective pixel is then replaced by a local 
brightness value obtained from the defective pixel's neighboring pixels. The process 
includes video subsampling, meaning that the defective pixel detection is carried out and 

25 repeated at a pre-specified frame rate to ensure optimum detection and correction at a 
minimal level of scanning. A statistical database is kept so that truly anomalous pixels 
can over time be distinguished from false detection of true anomalies in the target image, 
lighting or other environmentally induced anomalies. 

The software algorithm is intentionally kept very simple so that the 

30 processor load and/or the video frame rate is minimally affected. First, the software 

algorithm is kept simple because it only performs three functions, namely the detection, 
correction and statistics functions. The statistics routine is kept simple because only 
position information and frequency of occurrence of anomalous pixels are tracked. 
Second, the software algorithm is kept simple so as to have a minimal impact on the rate 



at which data is transferred to the host, so that while a frame is being scanned for 
defective pixels, subsequent frames are not held up. The maximum impact of the 
software algorithm is to at worst reduce the video data transfer rate from 10 frames per 
second (fps) to 9 fps. Third, the software algorithm is kept simple such that the host 
5 processor's load is not increased by more 1 % when executing the algorithm. The 
anomalous pixels are detected in one frame and the corrections are later instituted on 
subsequent frames as they are read off the image sensor. 

Dynamic defective pixel detection and correction is important for a 
number of reasons. First and foremost, image quality can be significantly improved and 

10 second, sensor cost can be substantially decreased by increasing the yield or usability of 
otherwise scrap sensors. Dynamic intelligent host software application based anomalous 
pixel detection and correction has several additional cost and quality benefits over the 
traditional hardware-based approaches. First, it reduces manufacturing cost by not having 
to add and program memory during product assembly and test. Second, it allows for 

1 5 further reductions in cost by not including the correction logic in the video processing 
ASIC. Finally, it allows for the correction of raw pixel data taken from sensor photosites 
which may become defective over time and with operating environment changes. 

BRIEF DESCRIPTION OF THE DRAWINGS 
20 FIG. 1 is a simplified block diagram of a system according to one 

embodiment of the present invention. 

FIG. 2 illustrates a simplified flow chart of a method of detecting and 
correcting defective pixels according to one embodiment of the present invention. 

25 DESCRIPTION OF THE SPECIFIC EMBODIMENTS 

FIG. 1 shows an embodiment of the present invention which uses software 
algorithms running on an intelligent host processor 1 14 to dynamically correct defective 
or anomalous pixels. In one embodiment, the intelligent host is a server or a PC. The 
system includes an image sensor 1 10 to capture an image, an intelligent host 101 having a 

30 processor 1 14 to execute the computer program stored in storage 118, and a system bus 
1 12 to facilitate device communications. The image sensor array can either be a charge- 
coupled device (CCD) or a complimentary metal oxide semiconductor (CMOS) image 
sensor array, which are typically parts of a video or still digital camera. The image sensor 
110 communicates with the host 101 via an external bus 111, which can be a universal 
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serial bus (USB) or a parallel port. Upon transfer of appropriate calls between the 
processor 114 and the image sensor 110, the computer program is loaded into memory 
116. 

The system works by reading off or scanning the raw image data from an 
5 image pickup device 1 10 for pixels that vary more than a specific amount in intensity 
from their neighboring pixels. The raw image sensor data is the unprocessed brightness 
data output obtained from the image sensor and which has not gone through any lossy 
compression or color processing. The image sensor reads analog voltage or current, 
converts to digital and sends the signal to a host with no further processing or 
10 compression. The photosites on the image sensor are used to capture either color or 
monochrome digital still or video images. The raw image data is sent to the intelligent 
1 3 host over a bus with a data transfer rate which is determined by the bus protocol of the 

p particular bus such as a universal serial bus (USB) or a parallel port. 

The raw image sensor data, the location and frequency of occurrence of 

: tit'-: ° 

1 *j j 15 each defective pixel, and the results of all intermediate computations performed by the 
: }S computer program are all stored in memory 116. Other algorithms are then used to 

j ; average the values of an anomalous pixel's neighboring pixels to replace the data from 
iU the defective pixel. The corrected data can then be further processed and ultimately 

ill! displayed on the monitor 120. The process includes video subsampling, meaning that the 

^j: 20 detection is carried out and repeated at various frame intervals. The video subsamphng is 
carried out on one of every 128 (1/128) frames. Alternately, the video subsampling can 
be carried out on every 1/64 video frames. And yet in an additional embodiment, the 
video subsampling is carried out on every l/(n times X) frames, where n is an integer and 
X is not equal to 50 or 60. 50 and 60 correspond to 50 Hz and 60 Hz, which are AC 
25 lighting frequencies used in the United States and Europe respectively. This way, it is 
ensured that anomalous raw data pixels are not artifacts of the artificial lighting systems. 
The use of video subsampling allows for rapid and optimum corrections without the need 
to scan every frame which would adversely impact the processing speed of the processor 
and the system. Naturally, no video subsampling is employed when detecting and 
30 correcting defective pixels in still images. 

A statistical database recording the location and the frequency of 
occurrence of defective pixels is generated and stored in memory 116 to allow the system 
to learn and adapt its operation over time. The trends from the statistical database are 
stored so that truly anomalous pixels can over time be distinguished from false detection 



of true anomalies in the target image, lighting or other environmentally induced 
anomalies. The operations logic of the statistical database and the details of the operation 
of the computer program are described below, 

FIG. 2 illustrates a simplified flow chart describing the functioning of the 
software program implemented in a system for detecting and correcting defective pixels 
according to one embodiment of the present invention. The first step in defective pixel 
detection and correction includes acquiring a portion of a frame of raw image data from 
an image sensor array, step 210. The raw image data may be data corresponding to a live 
scene being digitized or it may be data corresponding to a calibration or "black" 
background image. The "black" background image may be obtained by reducing the 
integration time of the sensor array or by reducing the gain of the image signal. The raw 
data from the "black" image can be used to detect over active photosites. The raw data 
corresponding to the "black" image must also be black, and if any pixel is not, then it 
corresponds to an overactive photosite. The use of the "black" image can enhance 
defective pixel detection by removing any brightness deviations that originate from the 
scene itself 

Although the data from an entire frame can be processed at one time for 
defective pixel detection and correction, the algorithm begins its function as soon as data 
from three lines from the image sensor has arrived at the host. Three lines of data from 
the image sensor array will allow for the processing of data for a pixel and all its 
surrounding neighbors. Moreover, when the sensor array is being used to capture video 
images, video subsampling is used so that not every frame of the video captured by the 
image sensor array is required for the defective pixel detection operation. In a preferred 
embodiment, defect detection is carried out at predetermined frame intervals as described 
above and the correction is applied to all video frames. The judicious use of video 
subsampling allows for the method to be very fast and efficient. 

In an alternate embodiment, no video subsampling is used such that the 
defective pixel detection is carried out on every video frame. The use of video 
subsampling is a function of the intelligent host's processor speed. If a processor is fast 
enough to allow detection of anomalous pixels in every frame, then video subsampling is 
not used. If, on the other hand, the processor is not fast enough to allow video processing 
at a desired frame rate, then video subsampling is used to ensure data transfer at that 
desired frame rate. Without video subsampling, anomalous pixel correction is immediate, 
such that defects are corrected in the very frame in which they are detected. With video 



subsampling, anomalous pixel correction is delayed until a frame is sampled to detect 
anomalous pixels. Therefore, the choice of using video subsampling, and the rate of 
subsampling are a function of the processor speed and a trade off between processor 
power and the delay before correction. 
5 After data acquisition, for each of the acquired pixels, the local pixel 

brightness value and also the average brightness value of all its immediately neighboring 
pixels are computed and stored, step 212. Next, a deviation threshold value is 
established, step 214. The deviation threshold establishes the acceptable level of variance 
between a pixel's brightness value and the average brightness value of all its immediately 

10 neighboring pixels. Next, for each acquired pixel, a local brightness deviation is 

computed, step 216. The local brightness deviation is the absolute value of the difference 
between a pixel's brightness value and the average of the brightness value of all its 
immediately neighboring pixels. 

Next, for each pixel whose data that has been acquired, its local brightness 

15 deviation is compared to the deviation threshold, step 218. Any pixel whose local 

brightness deviation exceeds the threshold deviation value is then flagged as a defective 
pixel. The physical location and the frequency of occurrence of each defective pixel is 
then recorded in a statistical database, step 220. The statistical database is then queried, 
to determine whether the defective pixel's data value should be corrected, step 222. The 

20 statistical database, by storing the location and frequency of defective pixels, develops 
over time trends which confirm which of the defective pixels warranted correction. The 
logic of the trends from the statistical database initially warrant correction of all flagged 
defective pixels as a default, and over time warrant pixel correction only if a particular 
pixel has an occurrence frequency of at least two out of the last four queries. 

25 The defective pixels that have passed through the statistical database filter 

are corrected next, step 224. In order to correct a defective pixel, the erroneous raw 
brightness data for a defective pixel is replaced by that pixel's local average brightness 
value, which is the average brightness value of all its immediately neighboring pixels. 
Next, the corrected data from the defective pixels as well as data from non-defective 

30 pixels is prepared to be sent for subsequent processing, step 226. Subsequent processing 
may include compression, color processing and encoding to data formats suitable for 
display. The defective pixel detection and correction is carried out on the raw data from 
the image sensor array because it is preferred to correct the data before any subsequent 
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processing has occurred since that processing itself can introduce artifacts which are hard 
to distinguish from artifacts which have been produced as a result of defective photosites. 

The software algorithm is intentionally kept very simple so that the 
processor load and/or the video frame rate is minimally affected. First, the software 
5 algorithm is kept simple because it only performs three functions, namely the detection, 
correction and statistics functions. The statistics routine is kept simple because only 
position information and frequency of occurrence of anomalous pixels are tracked. 
Second, the software algorithm is kept simple so as to have a minimal impact on the rate 
at which data is transferred to the host, so that while a frame is being scanned for 

10 defective pixels, subsequent frames are not held up. The maximum impact of the 

software algorithm is to at worst reduce the video data transfer rate from 10 frames per 
second (fps) to 9 fps. Third, the software algorithm is kept simple such that the host 
processor's load is not increased by more 1 % when executing the algorithm. The 
anomalous pixels are detected in one frame and the corrections are later instituted on 

15 subsequent frames as they are read off the image sensor. 

Therefore, when the system is operating, as raw image data from the image 
sensor is obtained and sent to be processed on a host, frames are captured for defective 
pixel detection and correction as described above. Once a defective pixel has been 
identified and its correction has been warranted by the statistical database, that correction 

20 will continue for all subsequent frames until the next query is made to the statistical 

database. If at that time the initially flagged defective pixel continues to be designated as 
a defective one, then the corrections will continue as described above. If on the other 
hand, the statistical database does not warrant the correction of defective pixels, then 
those pixels will no longer be corrected until the next query to the statistical database is 

25 made. The dynamic nature of this system will also allow for corrections of data from 
photosites that become defective over time due to post-manufacture environmental 
changes. 

The statistical analysis segment of the defect detection and correction 
algorithm is an optional one. It is aimed at increasing the efficiency of the pixel 
30 corrections, so as not to correct anomalies that were not caused by defective photosites, 
and hence save processing time and load. However, the gains in efficiency must be 
balanced against the load imposed by the statistical analysis portion itself. In an alternate 
embodiment not employing the statistical analysis portion, all pixels that get flagged as 
defective get corrected. As in the choice of using or not using the video subsampling, the 



decision to employ the statistical analysis portion of the defect detection and correction 
algorithm depends on a trade off between efficiency and processor power. If the 
processor is fast enough, then efficiency concerns are not so important. On the other 
hand, if the processor is not fast enough, then pixel correction efficiency becomes 
5 important enough to warrant the implementation of the statistical analysis portion of the 
defect detection and correction algorithm. 

As will be understood by those of skill in the art, the present invention 
may be embodied in other specific forms without departing from the essential 
characteristics thereof. For example, the rate of video subsampling could be varied, or 
10 not done at all. Alternately, the logic of the optional statistical database could be altered 
from one of correcting defective pixels as a default to one where defective pixels are 
corrected only if warranted by the trends from the statistical database. Accordingly, the 
foregoing disclosure in intended to be illustrative, but not limiting, of the scope of the 
invention which is set forth in the following claims. 

15 
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WHAT IS CLAIMED IS: 



1 


1 . A method of detecting and correcting defective pixels in raw data 


2 


taken from an image sensor used to obtain a digitized image, wherein said raw data 


3 


includes normal pixels and defective pixels, said method comprising the steps of: 


4 


(a) receiving a raw data signal for each pixel in said image; 


5 


(b) computing for each pixel received from said image sensor a brightness 


6 


value; 


7- 


(c) computing for each pixel received from said image sensor a local 


8 


brightness value; 


9 


(d) computing for each pixel received from said image sensor a local 


, h 10 

PS" 


brightness deviation of said brightness value from said local brightness value; 


1 11 


(e) setting a deviation threshold; 


?!• 12 


(f) comparing for each pixel received from said image sensor, its local 


T;i 13 


brightness deviation to said deviation threshold and designating pixels having local 


14 


brightness deviations greater than said deviation threshold as defective pixels; 


; AD 


( rj\ r^nrrlina the Inr^tinn nf <;aid defective nixels in a statistical database! 


S i 6 


(h) recording the frequency of occurrence of said defective pixels in said 


A; 17 


statistical database; and 


I l g 


(i) correcting the brightness value of said defective pixels, provided said 


0 19 


correcting is warranted by trends from said statistical database. 


1 


2. The method of claim 1, wherein said local brightness value is the 


L 


cn-ifhm^tir' Q\rp>ra cr^ nftTie htr\ crhtnpc:*? vainer nf all nixels immediatelv neighboring and 

ell llllLIlC HO cl V CI ctciC Ul LI 1C- Ullsilllllwoo Vdltiv^o \J x. an jpjL-^wxo iiinii^uiuuvij t> XA1 -' ^ A o 


3 


surrounding said pixel. 


1 


3. The method of claim 1, wherein said local brightness deviation is 


2 


the absolute value of the difference between said pixel's brightness value and said pixel's 


3 


local brightness value. 


1 


4. The method of claim 1, wherein said correcting is achieved by 


2 


replacing said defective pixel s bngtitness value by said detective pixel s local ongnxnebs 


3 


value. 


1 


5. The method of claim 1, performing said detecting and correcting of 


2 


said defective pixels dynamically and without any operator intervention. 
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1 6. The method of claim 1 , wherein said image sensor is a part of a 

2 digital video camera. 

1 7. The method of claim 1 , wherein said image sensor is a part of a 

2 digital still camera. 

1 8. The method of claim 1 , wherein said image sensor is one of (a) a 

2 charge-coupled device (CCD) image sensor array and (b) a complimentary metal oxide 

3 semiconductor (CMOS) image sensor array. 

1 9. The method of claim 1 , wherein said raw data is the unprocessed 

2 brightness value data which is output by said image sensor which has not gone through 

3 either lossy compression or color processing. 

1 10. The method of claim 1 , performing said detecting and correcting 

2 on a portion of said raw data obtained from said image sensor array corresponding to a 

3 portion of a frame of a video image. 

1 11. The method of claim 1 , performing said detecting and correcting 

2 on a portion of said raw data obtained from said image sensor array corresponding to a 

3 portion of a still digital image. 

1 12. The method of claim 1, wherein said statistical database, by storing 

2 the location and frequency of defective pixels, develops over time trends which confirm 

3 which of said defective pixels are warranted for pixel correction, wherein said trends 

4 initially warrant pixel correction as a default and over time warrant pixel correction only 

5 if a particular defective pixel has an occurrence frequency of at least two out of four 

6 queries. 

1 13. The method of claim 1, wherein said detecting includes video 

2 subsampling, wherein using video subsampling said detecting is carried out on video data 

3 frames at a rate between one of every 128 video frames and 1 of every 32 video frames, 

4 and wherein said correcting is continuous on every video data frame. 

1 14. The method of claim 1 , wherein said detecting includes video 

2 subsampling, wherein using video subsampling said detecting is carried out on video data 
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3 frames at a rate of one of every n times X frames, where n is an integer and where X is 

4 not equal to either 50 or 60. 

1 1 5. A dynamic method requiring no user intervention for detecting and 

2 correcting defective pixels in raw data taken from an image sensor which is part of one of 

3 (a) a digital video camera and (b) a digital still camera, used to obtain a digitized image 

4 which is sensed by a camera and transmitted over a bus to a PC, wherein said raw data 

5 includes normal pixels and defective pixels, said method comprising the steps of: 

6 (a) receiving raw data signals for each pixel from said image; 

7 (b) computing for each pixel received from said image sensor a brightness 

8 value; 

9 (c) computing for each pixel received from said image sensor a local 

10 brightness value, wherein said local brightness value is the arithmetic average of the 

1 1 brightness values of all pixels immediately neighboring and surrounding said pixel; 

12 (d) computing for each pixel received from said image sensor a local 

13 brightness deviation of said brightness value from said local brightness value, wherein 

14 said local brightness deviation is the absolute value of the difference between said pixel's 

15 brightness value and said pixel's local brightness value; 

16 (e) setting a deviation threshold; 

17 (f) comparing for each pixel received from said image sensor, its local 

1 8 brightness deviation to said deviation threshold and designating pixels having local 

19 brightness deviations greater than said deviation threshold as defective pixels; 

20 (g) recording the location of said defective pixels in a statistical database; 

21 (h) recording the frequency of occurrence of said defective pixels in said 

22 statistical database; and 

23 (i) correcting the brightness value of said defective pixels, provided said 

24 correcting is warranted by trends from said statistical database, wherein said correcting is 

25 achieved by replacing said defective pixel's brightness value by said defective pixel's 

26 local brightness value, wherein said statistical database warrants pixel correction if a 

27 particular defective pixel has an occurrence frequency of at least two out of four queries; 

28 and 

29 wherein said detecting is carried out on video data at a rate of one of (a) 

30 between one of every 128 video frames and 1 of every 32 video frames, and (b) one of 
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3 1 very n times X frames, where n is an integer and X in not equal to either 50 or 60, and 

32 where said correcting is carried out continuously on every video data frame. 

1 16. A system for detecting and correcting defective pixels in raw data 

2 taken from an image sensor used to obtain a digitized image, wherein said raw data 

3 includes normal pixels and defective pixels, said system comprising: 

4 (a) an image sensor to record an image of a scene, said image sensor 

5 containing a grid of photo sites to convert light shining on said photosites to electrical 

6 charges, wherein said electrical charges are converted to a series of analog charges which 

7 are then converted to digital signals by an analog to digital converter when said image is 

8 read off said sensor and; 

9 (b) an intelligent host configured to receive said digital signals from said 

10 image sensor, wherein said intelligent host a computer program product comprising: 

11 (i) a computer usable medium having computer readable code embodied 

12 therein for causing the detection and correction of said defective pixels, said computer 

1 3 program product comprising: 

14 (1) computer readable program code devices configured to cause a 

15 computer to receive a raw data signal for each pixel in said image; 

16 (2) computer readable program code devices configured to cause a 

17 computer to compute for each pixel received from said image sensor a brightness value; 

18 (3) computer readable program code devices configured to cause a 

19 computer to compute for each pixel received from said image sensor a local brightness 

20 value; 

21 (4) computer readable program code devices configured to cause a 

22 computer to compute for each pixel received from said image sensor a local brightness 

23 deviation of said brightness value from said local brightness value; 

24 (5) computer readable program code devices configured to cause a 

25 computer to set a deviation threshold to be used in conjunction with said local brightness 

26 deviation; 

27 (6) computer readable program code devices configured to cause a 

28 computer to compare for each pixel, its local brightness deviation to said deviation 

29 threshold and designate pixels having local brightness deviations greater than said 

30 deviation threshold as defective pixels; 
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(7) computer readable program code devices configured to cause a 
computer to record the location of said defective pixels in a statistical database; 

(8) computer readable program code devices configured to cause a 
computer to record the frequency of occurrence of said defective pixels in said statistical 
database; and 

(9) computer readable program code devices configured to cause a 
computer to correct the brightness value of said defective pixels, provided the correction 
is warranted by trends from said statistical database. 

17. The system of claim 16, wherein said image sensor transmits said 
digital signals to said intelligent host via a bus, wherein said bus connects said image 
sensor to said intelligent host. 

18. The system of claim 16, wherein said intelligent host is a server. 

19. The system of claim 16, wherein said intelligent host is a personal 

computer. 

20. The system of claim 16, wherein said local brightness value is the 
arithmetic average of the brightness values of all pixels immediately neighboring and 
surrounding said pixel. 

21. The system of claim 16, wherein said local brightness deviation is 
the absolute value of the difference between said pixel's brightness value and said pixel's 
local brightness value. 

22. The system of claim 16, wherein said correction is achieved by 
replacing said defective pixel's brightness value by said defective pixel's local brightness 
value. 

23. The system of claim 16, wherein said image sensor array is one of 
(a) a charge-coupled device (CCD) image sensor array and (b) a complimentary metal 
oxide semiconductor (CMOS) image sensor array. 

24. The system of claim 16, wherein said raw data is the unprocessed 
brightness value data which is output by said image sensor which has not gone through 
either lossy compression or color processing. 
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25. The system of claim 17, wherein said bus is one a (a) universal 
serial bus (USB) and (b) a parallel port. 



1 26. The system of claim 16, performing said detecting and correcting 

2 of said defective pixels dynamically and without any user intervention. 

1 27. The system of claim 16, wherein said image sensor is a part of a 

2 digital video camera. 

1 28. The system of claim 16, wherein said image sensor is a part of a 

2 digital still camera. 

1 29. The system of claim 16, performing said detecting and correcting 

2 on a portion of said raw data obtained from said image sensor array corresponding to a 

3 portion of a frame of a video image. 

1 30. The system of claim 16, performing said detecting and correcting 

2 on a portion said raw data obtained from said image sensor array corresponding to a 

3 portion of a still digital image. 

1 31. The system of claim 1 6, wherein said statistical database, by 

2 storing the location and frequency of defective pixels, develops over time trends which 

3 confirm which of said defective pixels are warranted for pixel correction, wherein said 

4 trends initially warrant pixel correction as a default and over time warrant pixel correction 

5 only if a particular defective pixel has an occurrence frequency of at least two out of four 

6 queries. 

1 32. The system of claim 16, wherein said computer program causes 

2 said detecting to be carried out on video data frames at a rate between one of every 128 

3 video frames and 1 of every 32 video frames, and said computer program causes said 

4 correcting to be carried out continuously on every video data frame. 

1 33. The system of claim 16, wherein said computer program causes 

2 said detecting to be carried out on video data frames at a rate of one of every n times X 

3 frames, where n is an integer, and where X is not equal to either 50 or 60, and said 
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computer program causes said correcting to be carried out continuously on every video 
data frame. 



1 34. The system of claim 16, wherein said computer program product 

2 consists of an anomalous pixel detection portion, an anomalous pixel correction portion 

3 and a statistical analysis portion. 

1 35. The system of claim 1 6, wherein execution of said computer 

2 program product does not increase processor load by more than between 1 percent to 80 

3 percent. 

1 36. The system of claim 16, wherein execution of said computer 

2 program product does not reduce video processing by more than 1 frame per second. 
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DYNAMIC ANOMALOUS PIXEL DETECTION AND CORRECTION 

ABSTRACT OF THE DISCLOSURE 
A method and a system for dynamically detecting and correcting anomalous 
pixels in the raw data taken from an image sensor array such as a CCD or a CMOS sensor 
5 array, thus allowing the use of dumb cameras to capture digital images for subsequent use by 
an intelligent host - such as being displayed on a computer monitor. This invention uses 
software algorithms running on an intelligent host processor to dynamically correct the 
anomalous pixels in the raw data taken from an image sensor array typical of those in a 
digital still or a video camera. Using the combination of a dumb camera which provides raw 
10 data to an intelligent host, which does all the subsequent image processing, the system works 
£ by scanning an image frame for pixels that vary more than a specified amount in their 
< P brightness value from their neighboring pixels and designating those as defective pixels. The 
P location and frequency of the photosites sending the defective pixels are stored in a statistical 
; J? database in the computer's memory. The brightness value of a defective pixel is then 
■,pL5 replaced by a local brightness value obtained from the defective pixel's neighboring pixels. 

The process includes video subsampling, meaning that the defective pixel detection is carried 
j y out and repeated at a pre-specified frame rate to ensure optimum detection and correction at a 
ifi minimal level of scanning. A statistical database is kept so that truly anomalous pixels can 
K over time be distinguished from false detection of true anomalies in the target image, lighting 
20 or other environmentally induced anomalies. 

PA 3036808 v3 
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